Frame interpolation creates an image frame from neighboring images. The neighboring images may be fields in an interlaced video format, used to form a frame of data, or neighboring frames of a soon-to-be-created frame.
In the simplest approach, one could increase the frame rate by repeating the most recent frame until the next frame is ready for display. However, this does not account for moving objects which may appear to jump from frame to frame and have flickering artifacts.
Motion estimation and motion compensation techniques may alleviate some of these issues. These techniques rely upon motion vectors to shift the image data for the moving object to the correct position in interpolated frames, thereby compensating for the motion of the object. Difficulties arise in the estimation of motion and the selection of the correct motion vector in the regions of the image where the moving object resides. These regions may have background areas that are initially uncovered in the background, but become covered by the object in motion. Similarly, these background regions may be initially covered by the object, and then become uncovered as the object moves away. In either case, selection of motion vectors becomes difficult. These regions will be referred to as ‘occluded.’
Motion estimation and motion compensation techniques may rely upon motion vectors to ensure that the interpolated frame's pixel data correctly depicts the motion of the object. Some approaches using motion vectors may determine motion vectors for blocks of pixels in the image data. The motion vector, previous frame and current frame data and occlusion information are used to interpolate the new frame.
At the boundary of objects, the correct motion vector for a particular pixel may actually reside in an adjacent block. For approaches that blend motion vectors from neighboring blocks, this may result in the wrong motion vector for several pixels. Boundaries are typically also occluded areas, so just selecting a motion vector having the lowest pixel difference between the two frames may not work. The pixel may be in a covered area in one frame and an uncovered area in the next frame, resulting in a large pixel difference. Selecting an improper motion vector affects the resulting image data.